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ABSTRACT 

Because of limited access to technical systems used 
to train men to maintain complex electronic equipment, an 
instructional format employing low-cost slides and photos was used to 
provide students with simulated experiences. Five experimental 
instructional blocks were developed and tested in tiro classrooms. . 
Results were compared with two control classes, and the experimental 
groups showed superior mastery of the skills contained in the five 
instructional blocks. This report describes the background of the 
project, the research design, and the analysis of the data. (EMH) 
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SUMMARY AND CONCLUSIONS 



MILITARY PROBLEM 



The U.S. Army Air Defense School has the responsibility of training 
men to maintain complex electronic equipment. Due to the critical 
nature of systems such as the Improved Hawk, there is limited access to 
equipment. Therefore, there is a need to develop training alternatives 
which require a minimum amount of time on the actual equipment. 



RESEARCH OBJECTIVE 



The research project was to determine whether low-cost simulation 
(i.e., photos and slides) used in conjunction with carefully engineered 
instructional strategies could improve training in maintenance of Air ^ 
Defense Systems when administered by U.S. Army Air Defense School (USAADS) 
instructors . 



METHOD 



Lesson plans and simulation materials were developed for five blocks 
of instruction on the Improved Hawk IHIPIR. These were used in two 
experimental classes and the results were compared with two control group 
classes. 



RESULTS 

The experimental classes demonstrated superior performance involving 
content of the five blocks on several measures: 

1) Successfully completing troubleshooting problems 
(simulated on equipment turned off) in the time 
allowed, with significantly fewer errors. 
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3) 



2) 



Demonstrating checks and adjustments faster, and 
with fewer errors; 

Better scores on quizzes which involved performance- 
oriented items. 



CONCLUSIONS 



The instructional strategies and simulation methods significantly 
J'jnproved training where real equipment was in short supply, within the 
practical constraints of the USAADS situation and with minimal increase 
in costs* 
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BACKGROUND 



MILITARY PROBLEM 

The Army must train men to maintain complex electronic equipment, 
with stringently limited time for practice on the real equipment. The 
training problems are especially severe with newly developed and complex 
equipment. One example of a critical system is the Improved Hawk System. 
Complex electronic simulators might provide the needed practice, but 
their cost and development time make this approach generally infeasible 
and unacceptable. Another constraint for the present research is that 
training must be accomplished in existing facilities by existing staff 
without extensive special training. 



RELEVANT STUDIES IN THE LITERATURE 

One promising approach to the lack of time on actual equipment is 
the use of very simple, low-cost trainers or mock-ups having only gross 
visual fidelity; for example, devices having dummy instruments, photos, 
or even line drawings. Studies of a variety of procedural tasks have 
demonstrated the effectiveness of such devices (Cox, et. al . , 1965; 
Denenberg, 1954; Dougherty, et. al . , 1957; Fox, et. al . , 1969; Grimsley, 
1969, a, b, c; Miller, 1958; Prophet and Boyd, 1970) • However, one 
should not presume that such simple devices would be effective for all 
types of training or in all training situations. As training becomes 
more advanced, the students become increasingly familiar with the equip- 
ment, and therefore, may no longer need visual reference to the equipment 
Also, there are increasingly complex logical inferences (cognitive re- 
quirements)- involved in troubleshooting, so perhaps there is relatively 
less need to establish visualization processes, even though visual cues 
continue to be involved in the task. 

Swanson (1954), using highly experienced electronic maintenance 
technicians as subjects, found no* differences among training aids with 
various levels of realism. The6e technicians could perform effectively 
from general verbal directions or abstract symbols, as they had been 
trained to do. In a follow-up experiment, Swanson, Lumsdaine, and Aukes 
(1956), using inexperienced airmen as subjects, found that training with 
a visual mock-up did lead to superior performance on a parts recognition 
test, as compared with those who had similar training without the visual 
mock-up. One would expect, therefore, that the effectiveness of visual 
mock-ups would depend upon the stage of training and the kind of per- 
formance tested. 
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In previous ARI-sponsored HumRRO research*, the instructional system 
fnr conventional Hawk training was analyze^ for influences on instructor- 
performance. Although guided by the Program of Instruction (POI) , lesson 
plans and vaull files, the instructor has considerable latitude in his-* • 
instructional approach, especially if he is a Primary Instructor. The ' 
system further tends to give more prestige to the platform instructor 
than to the practical exercise instructor. There is also little incen- 
tive for an instructor to disseminate apparently effective instructional 
strategies to others. There appears, therefore, to be a definite need 
for preparing instructors in performance-oriented instruction. 

As part of INTERFACE, an experimental Instructor Workshop in Func- 
tional Context Training was conducted. In the Workshop, it appeared that 
instructors could learn to use a photo mock-up with skill and ingenuity, 
although at first they seemed to have considerable difficulty in faith- 
fully following a job sequence in their instruction. 

I 

RESEARCH OBJECTIVE \ 



The research reported here was designed to answer the following 
question: 

Can low-cost simulation (i.e., photos and slides) improve 
instruction in electronic maintenance on complex systems, 
such as Improved Hawk (in which real equipment is scarce) , 
when used by typical USAADS instructors following carefully 
engineered instructional strategies? 



*HumRRO Work Unit INTERFACE 
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SUBJECTS 



• The subjects were 78 students enrolled in four successive classes 
of the Improved Hawk Firing Section Mechanic (24C20) Program of Instruc- 
tion (POI). The POI covers: (1) the Launcher, and (2) the Improved 
High Power Illumination Radar (IHIPIR) . The first two classes (75-5, 
N«23 and 75-6, N=16) were pooled to form the control group, and the other 
two classes (75-7, N=22 and 75-8, N=17) were pooled to form the experi- 
mental group. 



BLOCKS OF INSTRUCTION (LESSONS) 



The research investigated effects of the methods applied to the 
following five blocks of instruction on the Improved High Power Illumi- 
nation Radar, IHIPIR (presented immediately after the Introduction): 

1. Energizing and Control Circuits, 8 periods* of 
classroom activity. 

2. Energizing and Control Circuits, 6 periods of 
practice on station with the equipment. 

3. AC and DC Power Distribution, 4 periods of 
classroom activity. 

4. AC and DC Power Distribution, 12 periods of 
practice with the equipment on station. 

5. Transmitter RF Generation, Arc Detection, and 

• • Modulator Bias Circuits, 8 periods of classroom 
activity. 

During the first four blocks of instruction the student is learning 
appearance and location of various components while performing the basic 
operations involved in energizing the equipment, following the basic 
conventions of their technical manuals. Only the material in the last 
block of instruction is considered difficult to understand, compared 
with typical blocks of instruction. The classroom activity is designated 



*A period is 45 minutes with a 10 minute break between periods. 
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"PE 3" which stands for "Practical Exercise, type 3", signifying 
intention of USAADS to make it as much like a job activity as pos 
rather than like the traditional lecture. 



TREATMENTS 
Control Group 

This section describes the general approach in the conventional 
classes, as a basis for comparison with the experimental treatments 
that will be described in the next sections. 

Classvoom (PE 3). The classroom work for the Control Group during 
the three PE 3 blocks involved theory, with large amounts of circuit- 
tracing using schematic diagrams. Circuit-tracing typically moved l^ft- 
^o-right as ?he instructor explained how the circuit "works" Various 
facts were mentioned as they applied to particular points. There were 
few if any visual aids beyond blackboard and chalk and schematic diaf fins 
Prior to this, the students have had six periods with the equipment (with 
only a part of that time spent close enough to the radar to become fami- 
liar with it). The average student had little more experience with the 
radar than the majority of readers of this report. Procedures were not 
described during these periods; rather, the "doing" was relegated to the 
work on station. 



On Station (PE 1). During the two PE 1 blocks the students learned 
to perform checks and adjustments and fault isolation. The Primary 
Instructor was assisted by three other instructors (each instructor had 
about five students working with one radar) . Each instructor works with 
one pair of students at a time: one student reads the procedure from 
the manual while the other does it. The other students were generally 
supposed to observe the practice. Sometimes the students not performing 
on the radar were instructed to perform various supplementary tasks at 
their tables while waiting for their turn on the equipment. ^PPf 
difficult for the instructors to devise really worthwhile activities for 
those not on the radar, since the students on the equipment had to be 
monitored closely for safety of personnel and equipment. 



Experimental Group; Instructiona l Strategies 

■/'^^ 

For the experimental classes new instructional, strate^iJSyit^ijjg 
devised. An instructional strategy is a general rule for ^ 
content information so that it is easier to learn and ^PPU J^^^^a^ti^.^- 
These rules cover sequencing of instruction and various o^li^r io^ 
Each instructional strategy tends to be especially appropi^-i^'c^ g. , 
particular purposes and for particular kinds of informat 1^,11 v^^e 
learning how to do a task, it is generally efficient to p)::^!'^^ 
steps in the task sequence) . 

Instructional strategies are commonly referred to as '''f^U'f Sp,,^ 
things in a logical order". Although that is true, it is h^^^^ 
cific enough to be of much help in designing instruction. 

Instructional strategies are often used in various cotjjbj^natiioj^ 
All the instructional strategies in this study use visual Sj^^^t th^> 
in combination with other strategies. Therefore, the eff^C a 
visuals were confounded with the other instructional stratus:".'' 
their independent effects can not be determined. 



Visual Simulation 

Classr-oom (PE 3). The training equipment for t^e exp^^j^n, 0^^ 
classes consisted of two large-scale color photos (60" x ^ J^VJ^S 
each end of the radar, and a series of color slxdes Th^ Sr^?""^^? 
the radar (depicted in Figure 1) contained almost all of ^^Jj^ ^ 
displays and controls (although sometimes these were loc^^^JJ f^h 
panel). As a general practice, the instructors were to b^JJV ^ot 
large color photos for orientation and then refer to the ^>^o 
details on each particular teaching point. 

The large color photos were hung at the front of the ^0^^^ ^^^^^^ 
of the chalkboard*, and were removed when necessary to u^^ y.'^laM 
board. Before class 75-8, in order to facilitate access ei^h^^^- 
board, they were mounted** on cork panels which slide in p^V^ 
side. This arrangement seemed satisfactory in all respecviP- 



*To avoid glare from ceiling lights, the panels were t^lt^ 
(about 6") at the top. 

**Mounting by a cord facilitated removal and storage. 
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Figure 1. The two ends of the radar depicted in the color photos. 
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For the unit on AC and DC Power Supplies there were color photos 
(10" X 12") of each Power Supply. These could be hung on the large 
photos in the proper locations by means of small screw heads protruding 
from the frame. 



On Station (PE 1). For each pair of students there was a set of two 
color photos like those used in the classroom by the instructor, but on 
a sbmewhat smaller scale (40" x 24" each). These were hung by pegboard 
hooks around the perimeter of each station. To represent the internal 
components there was a looseleaf notebook provided for each student with 
line drawings of the equipment located behind each of the panels, numeri- 
cally coded to the panels. (Students tended to keep these notebooks 
available because the looseleaf covers were also handy to hold their checl 
manuals.) Synchronized tape/slide equipment was also available. 



Structuring the Lessons 



The instructional strategies concern the structure inherent in the 
lessons. The structure Is most apparent in the sequence of presentation, 
although it may be also a matter of emphasis and other aspects of style 
of presentation. In the instructional blocks studied, the most important 
structures to be considered are the following; 



Jol Situation and Job Sequence. This is a matter of presenting 
information as it is to be used on the job, most notably in the sequence 
of job performance. (However, .it may be only a segment of a task to be 
performed.) This is often called the "functional context method of 
presentation. 

The job sequence in classroom presentations (along with using visual 
simulation) was used extensively. This was in contrast to the conven- 
tional lesson plans, which generally discussed the circuits in the sequence 
they appeared in the manuals. 

The most common use of the "job situation" strategy was in discussing 
the functional diagrams. First, the checks and adjustments would be 
demonstrated and discussed, using the color photos and slides, showing 
how a trouble would lead to the Fault Isolation Procedures (FIP)- Then 
the FIP would be demonstrated and discussed, showing how that delineates 
a circuit in the schematics within which the fault must lie. Then the 
circuit is traced and discussed, but (unlike the conventional treatment) 
in the context of looking for a trouble . 



Identify End Points First in Schematics. This is a matter of locating 
the first and last functional reference points in the schematics before 
tracing the rest of the circuit. Thus, the goal of the tracing is identi- 
fied first. 

Relating Elements in Schematic Diagrams to the Equipment. In the 
above strategy, noting end points also served to relate the schematics 
to the real equipment, since the end point almost always was observable 
on the equipment (e.g., a meter). The slides also provided opportunity 
for relating points in the schematics to their visual appearance. 

Grouping. This is the strategy of sorting the subject matter into 
natural categories. It is commonly called the "whole-to-part method. 
In introducing the general categories, it is important to avoid vagueness. 

In the psychological literature such grouping is called "chunking" 
or "advance organizers". The- human mind can only think about a few items 
at once (this is the limit of immediate memory, or what a person is con- 
sciously aware of at any particular moment). It seems that a P^^o" 
only hold about seven items (technically called "chunks " of information) 
in Its immediate memory at one time (G. A. Miller. 1956) ^^f.^^^^J;.. 
if a person looks up an unfamiliar telephone number (? digits or chunks 
of information) , he can probably remember it until he dials it ; but if 
someone interrupts him by asking for some other ^^^5,f ^^^^ 

to "push" some of the digits out of his memory and he will probably have 
to look it up again. 

The implication for instructional practice is that the general cate- 
gories (the "whole") should be subdivided in stages, with fewer than 
seven subdivisions at any particular stage (generally three to five sub- 
divisions are effective). A common mistake in applying the whole-to-part; 
strategy is to jump from the general topic down to the detailed level, 
so that the learner can not relate the two levels. 

A good- example of the "grouping" strategy occurs in the first experi- 
mental block of instruction. Step 3 of the FIP concerns various events 
that might prevent the radar from going into "Standby". This generally 
is a matter of an interlock switch being open or faulty. This is not^ 
apparent to the novice until he gets x^ell into the 14 substeps of Step 3, 
wh?ch are spread over two large (11" x 16-1/2") P^8^l,f J^^'/ . 

interlock switches are checked out with a selector sv^tch indicator 
light at each side of the radar. The first cluster (or group) of substeps 
is sLp 3a in which the indicator lights themselves are checked to be sure 
that they will give the proper indications. The second cluster covers 
S^^ps S^thriugh 3g in which the interlocks on the ^ransmitter^side^are 
checked out by a similar switch. The third cluster i^'^l"'^%^J^^PJ/^_^ 
through 3k in^hich the interlocks on the Radar Set Group (RSG) side are 
checked out by a similar switch. The last cluster. Steps 3L through 3n, 



covers .the remaining components that might cauiie trouble between the 
interlocks and the ''Standby" indicator lights, including relays and a 
holding circuit. The students were directed to mark these clusters of 
steps in their manuals. Note that the grouping is not arbitrary ^ and 
that it corresponds precisely to visual aspects of the equipment (the 
switches and indicator lights) . 



Basic Case and VariaHons. It is often easier to remember several 
similar procedures if they are thought of simply as variations of one 
basic method. That way the person has many fewer items to learn. When 
the variations differ in complexity, the simplest is generally taught 
first (the so called ''simple- to-complex" strategy). 

This is illustrated in the above example of clustering substeps 
within the second and third clusters. The substeps in each of these 
clusters is identical in preparation and in holiing the Standby button 
depressed, differing only in the particular position of the selector 
switch. 



Causal Sequence. With this strategy a cause-and-eff ect chain of 
events is described in sequence. Note that this is not necessarily the 
same as a time sequence because the events may occur simultaneously or 
continuously (e.g., in a system of mechanical linkages the whole set 
moves together, although each link causes movement in the next one it 
touches, going from input to output motion). 



Left-to-I^ight. This was not one of the experimental instructional 
strategies but is mentioned because it is perhaps the most commonly 
encountered arbitrary strategy. With this strategy the items are described 
in a sequence moving from lef t-to-rifiht , as they occur* Por. Instance, 
an instructor may describe the indicators and controls on an item of 
equipment "starting from the upper left-hand comer and moving across^ 
the panel". The strong "left-to-right" tendency is based upon people s 
reading habits. Although it is sometimes appropriate (e.g.,. reading 
schematics), it very often is not. Therefore, whenever om. counters 
the left-to-right sequence in instruction, one should anal> . the sit- 
uation to determine whether it is an appropriate strategy. 



Experimental Group Instruction 

Classroom (PE 3)_ 

Enevqizing and Control Circuits (8 Periods). Here the predominant 
instructional strategy was the job sequence of checks and adjustments 
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and Fault Isolation Procedures (FIP) , illustrated with the large color 
photos and a series of slides. This began with a short series of checks 
and adjustments (Steps 1-3, Table 3-8 in TM 9-1430-533-12-1). Then each 
step was repeated, assuming there was a fault, which led to a correspond- 
ing step in the FIP, and using the functional schematic as specified. 
In Step 3 of the FIP the instructional strategy of grouping the substeps 
was also employed (see the lesson plan in Appendix A, Vol. II, for more 
details of the presentation). 

In order to further assist students in finding components of ths 
equipment, two locator diagrams, developed by Raytheon, were handed out. 
Using these diagrams, the student could locate -'ti item from knowing either 
the title of a block on the schematics, or the numerical code that appears 
by that title. 

The instructor seemed to have little difficulty in adopting the 
methods. In his presentation he referred repeatedly to the color photos 
and to the slides, as required by the lesson plan. There did not appear 
to be. any tendency to *'taper off* and return to traditional methods. 

AC and DC Power Distribution (4 Periods). The prominent instructional 
strategy here was grouping of power supplies supplemented by job sequence 
within each group. 

The power supplies were divided into four functional groups, as 
described in a handout (Appendix B, Vol. II). It seemed that a major 
difficulty in learning to work with the various power supplies was the 
confusion resulting from their nine locations, and each location supply- 
ing multiple voltages. The grouping is based upon the BITE (built-in 
test equipment) their location, and their function in the system. The 
grouping corresponds rather closely with the basic block diagram of the 
system (Figure 24-2 in TM -9-1430-533-12-2-2) • 

As an overview, the instructor demonstrated where checks were to be 
made on each group of power supplies, located the individual power sup- 
plies within the group, and shoxc^ed slides of them. Then, as each power 
supply group x^ras discussed in detail, the color photos of that group were 
to be hung near their locations on the large photos and the photos from 
the previously discussed group removed so as to distinguish sharply which 
group was being discussed. 

In both experimental classes the instructor apparently had some dif- 
ficulty in hanging and removing the photos of the power supplies according 
to plan. Sometimes pictures of previously discussed power supplies would 
be left in place. Sometimes the pictures within a group were hung one- 
by-one rather than all at once. Therefore, the groups may have been less 
sharply delineated than intended in the lesson plan. 

17 
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EF Generation^ Arc Detection^ and Modulator Bias (8 Periods). In 
this presentation the instructional strategy was based upon the under- 
lying causal sequence (as was the conventional lesson plan) supplemented 
by visuals showing the physical components. In this instance the Master 
Oscillator generates the RF energy into a waveguide, which is a rectangu- 
lar metal tube. The energy passes through various components that envelop 
or abut the rectangular waveguide; these components monitor, rotate, 
modulate, or otherwise process the RF energy. These are represented 
schematically in Figure 24-17 of TM 9-1430-533-12-2-2. In performing FIP, 
the student must relate these schematics to the equipment. One of the 
difficult things to teach about the RF Generation was that the process 
could be monitored in only a few places, although this generally was 
rather obvious from the physical appearance (but not from the schematics) . 
The points for monitoring show up as test jacks, crystal mounts for BITE, 
or other distinctive components. After the RF Generation chain was de- 
scribed completely, the specialized circuits for Arc Detection and Modu- 
lator Bias V7ere described and illustrated. 



Chi Station (PE 1) 



Tape/slide presentations demonstrating checks and adjustments were 
developed in multiple copies. (One presentation was for Energizing and 
Control Circuits, one for AC and DC Power Distribution, and one for RF 
Generation, Arc Detection, and Modulator Bias, although the latter was 
not part of the experimental units of instruction.) Each tape/slide 
demonstration was shoxm once to all students in another room. Each demon- 
stration consisted of reading the check manual with the slides demonstra- 
ting performance of the procedures. 

A series of FIP problems was developed. for use with the color photos 
and given to each pair of students. The original series of seven prob- 
lems (for class 75-7) was expanded to twenty problems for class 75-8. 
The directions and problems are given in Appendix C (Vol. II) • These 
problems present some critical information and approaches to FIP. How- 
ever, there were difficulties in administering these problems so that 
effective practice conditions were assured. First, there was no way 
devised to insure that the students actually performed the problems con- 
scientiously. Also, there was no way to insure that the students referred 
to the color photos when equipment was involved in some step. Special 
evaluation devices and techniques that were considered (such as an answer 
$heet on which erasures reveal the correctness or error of each response) 
seemed to entail more drawbacks than advantages. The resulting instruc- 
tional procedure appeared to be rather haphazard. 
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INSTRUCTOR SELECTION AND PREPARATION 



The instructor for the experimental classes was the instructor 
routinely assigned to teach class 75-7. The instructional strategies 
were explained to him in a briefing which emphasized visual simulation 
and job sequencing. The detailed methods to be followed were described 
in lesson plans (see Appendix A, Vol. II) , prepared in the format the 
instructor was used to. The new methods v/ere practiced in a dry run, 
with occasional comments, coaching, and encouragement by the Project 
Director, The Lesson Plans in Appendix A (Vol. II) reflect minor cor- 
rections and adjustments, which were requested by the instructor. A 
second series of dry runs was conducted similarly in preparation for 
class 75-8. 



CRITERION TESTING 



Routine Examinations 

The regular examination over the first four experimental blocks of 
instruction was one troubleshooting problem. The student was graded 
not only on whether he found the fault, but also on the procedure he 
used. Such testinj^ was insufficient for research purposes for several 
reasons. First, the testing was designed to discriminate at the lower 
end of the distribution, in order to determine whether a student would 
pass or fail, and hence, the scores tended to pile up at the upper end 
of the distribution, with the typical student getting a near-perfect 
score. Consequently, these exams did not provide much room for improve- 
ment for the average or superior student, so they have limited power for 
detecting differences in experimental training methods. Also, the prob- 
lems were not standardized; even within a test, an examiner would change 
problems if he had reason to suspect that the students who had taken the 
exair. were telling the others about it. Finally, a single problem does 
not =Drovide enough information to evaluate a progran?.. 



Experimental Examinations 



For the above reasons, an experimental testing program was institut 
The normal resting program was scheduled for an eight-period block of/ 
instruction during which the students were tested individually. Afpr 
taking their official examination, each student in both experimental an 

19 



12 



control classes was given another examination. The examiners were 
regular Army enlisted instructors assisted by contractor personnel who 
kept time and helped maintain uniformity of procedure. 

The experimental examination procedure consisted of three parts: a 
Practical Examination on the equipment, a Written Multiple Choice Exam- 
ination, and an Attitude Questionnaire (the corresponding forms are in 
A-ppendices D, E, and F respectively, Vol. II). 

Tractiaal Excwiincitions . During the Practical Examination the equip- 
ment was turned off and the students actually manipulated switches and 
pointed to dials, describing what they would expect normally. The first 
item was to perform all relevant checks and adjustments, followed by two 
Fault Isolation problems (one for Energizing Circuits, one for AC and DC 
Power Supplies). Directions are in Appendix D, Vol. II.' For the checks 
and adjustments, each student was given a maximum of 10 minutes to complete 
Tables 3-8, 3-9, and 3-10 of TM 9-1430-533-12-1. This resulted in rank 
order data, depending upon how many steps they completed, or if they com- 
pleted the entire procedure, how fast they completed it. The number of 
errors were also noted. For the Fault Isolation problems, each student 
was allowed five minutes for each of the two problems. The instructor 
wnuld provide feedback verbally at each step. One problem concerned an 
interlock (in the Energizing and Control Circuits) and the other involved 
a defective Power Supply (from the lessons on AC and DC Power Distribution^ 
The resulting data are whether each student successfully completed the 
problem in the 5 minutes allowed, and the number of errors made. 

On the Transmitter RF Generation, Arc Detection and Modulator Bias 
Circuits, the testing was done in conjunction with another Practical 
Examination later in training. That experimental testing consisted, of 
one representative check procedure and ono. relevant proDlera in trouble- 
shooting. For the check procedure, each student was given 8 minutes to 
complete' Step 19 of Table 3-11. This yielded rank order data, plus a 
tally of errors, as in the earlier check procedure. For the Fault Isola- 
tion problem, each student was allowed 7 minutes, and it was noted whether 
the problem was completed successfully, and the number of errors made. 



fizzes. During the same examination period that each class was 
given their Practical Examinations, they were also given short quizzes 
(Appendix E, Vol. II). The items in these qr-izzes were designed to 
involve some critical element of job performance. One quiz covered 
Energizing and Control Circuits. Class 75-5 took a completion form of 
this test, which proved difficult to score objectively; hence, they were 
not included in the analysis of quiz scores for Energizing and Control 
Circuits. Ths quiz on Energizing and Control Circuits consisted of 
Identifying the nine locations of the power supplies. 
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Opinion Questionnaires. To supplement the test data, questionnaires 
were administered to all classes (Appendix F, Vol, II), in which the 
students rated the general effectiveness of instruction. One section, 
administered to t'le Experimental Group only, asked for absolute judgment 
of the effectiveness of the visual aids and FIP problems. 
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ANALYSIS 



COMPARABILITY OF GROUPS 



• The comparability of the Experimental and Control Groups was first 
established by comparing them on the E.L. .Score of the Army Classifica- 
tion Battery (ACB) by V7hich they were qualified for the 24C20 POI. 



DATA SPECIALLY COLLECTED 

Practical Examination Scores 

Cliecks and Adjustments. Here the data consisted of rank-ordered 
scores, based upon speed in completing the checks (differences tested 
by Mann-VJhitney "U") and the number of errors in performing them 
\ (differences tested using x^) • 

Troubleshooting. There were two FIP problems administered on the 
first examination day, and one on the second. It was noted whether 
each person successfully completed each problem in the time allotted, 
and the number of errors. The statistic was used to test signifi- 
cance of both completion and errors. 



Quizzes 

Differences between group means for each of the three short quizzes 



were 



tested using the "t" test. 



Opinion Questionnaires 

Only descriptive data were computed for the student questionnaires. 
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RESULTS 



COMPARABILITY OF GROUPS. 



. On the E.L. Score of the Army Classification Battery (for those scores 
available), the mean for classes 75-5 and 75-6 (Control Group) was 112.25 
(n = 39), and for classes 75-7 and 75-8 (Experimental Group) was 112.-77 
(n = 27). The very small difference was xvrell within the range that would 
be expected by chance alone. It is assumed, therefore, that the groups 
were comparable. 



ROUTINELY ADMINISTERED EXAMINATIONS 



The results -of the first school practical- examination were highly 
skewed, with the majority scoring above 90%. The median score (50th 
percentile) for classes 75-5 and 75-6, taken together was 96 out of li 
compared v:ith 95 for the experimental classes (75-7 and 75-8). The 
difference vas not significant. 



EXPERIMENTAL PRACTICAL EXAMINATION 



Checks and Adjustments 

The classes were compared on rank-order data, derived from the speed 
with which they performed the procedures specified in Tables 3-8, 3-9, ' 
and 3-10 (TM 9-1430-533-12-1). An appropriate statistic is the Mann- 
\-Ihitney "U" (Siegel, 1956, pp. 116-126) which tests whether one group 
ranks significantly higher than another when the two are combined in a . 
rank-order distribution. 

\,men the Experimental Group (classes 75-7 and 75-8) was compared with 
the Control Group (classes 75-5 and 75-6) , the Experimental classes scored 
significantly better (z = 5.89, p < .001). Only four men in the Experi- 
mental Group scored below the median for the Control Group; no one in the 
Control Group scored above the median for the Experimental Group. This 
was a cumulative effect over the first four experimental blocks of instruc- 
tion, and the separate effects of particular blocks could not be isolated. 
The Experimental Group also tended to make significantly fewer errors ; . 
(Table 1, x\ = 11.08, p < .01). 



On the second Practical Examination covering RF Generation (Step 19 
of Table 3-11, TM 9-1430-533-12-1) almost all of the students did finish 
within the time limit (8 minutes) so the data were rank-order based upon 
time to f iniahv ' The Experimental Group was- significantly faster (z = 2.28, 
p < .05): This difference was much smaller than the corresponding one in 
the previous test on checks and adjustments, although still significant, 
statistically.- The median score for the Experimental Group was near the 
70th percentile for the Control Group. The differences in number of errors 
(Table 1) Xv'as not significant. 



Troubleshooting • 

During the first Practical Examination each student also performed 
two troubleshooting problems on the equipment with the instructor verbally 
supplying feedback. (See Appendix D, Vol. II, for details.) 

On the first problem 34 out of 39 students in the Experimental Group 
completed the problem successfully xvhile only 20 out of 39 in the Control 
Group (classes 75-5 and 75-6) completed it. (Results in Table 2.) This 
difference was significant statistically (x^ = 13.2, p < .001). 

On' the second problem the Experimental Group also performed better: 
26 out of 39 completed it, compared with 11 out of 39 in the Control Grou^ 
(X^ - 10.56, p < .001). They also made significantly fewer errors (x 2 " 
15.^92, p < .001). 

On the problem in the RF Generation system (Table 3), the Experimental 
Group tended to finish somewhat more often (x^ = 5.3, p < .05), although 
there was no significant difference in the number of errors made. 



Quizzes 



On the quiz covering Energizing_and Control Circuits, the Experimental 
Group scored significantly better (X = 5.23 vs. 4.22, t = 4.372, p < -OOl) 
than the Control Group (class 75-6 only, as noted previously under METHOD). 
Similarly, on the quiz on locations_of the power supplies, the Experimental 
Group scored significantly better (X = 7.82 vs 5.94, t = 4.67, p < .001). 

On the quiz covering RF Generation, Arc Detection, and Modulation Bias, 
the Experimental Group also tended to score somewhat better (X = 9.35 vs. 
7.42, t = 3.24, p < .01). 
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TABLE 1. EXPERIMENTAL 'PRACTICAL EXAMINATION, 
ERRORS ON CHECKS AND ADJUSTMENTS 







0 


1 


2 or more 


Tables 3-8, 3-9, 3-10. 
(^22 = 11.08, p < .01) 


Experimental Group 


14 


9 


16 


Control Group 


3 


7 


29 






0 


1 


2 or more 


Table 3-11, STEP 19. 
(x22 = 1-30, n.s.) 


Experimental Group 


7 


15 


17 


Control Group 


11 


13 


14 



25 

18 



TABLE 2. PERFORMANCE ON FAULT ISOLATION PROBLEMS 
IN. FIRST EXAMINATION PERIOD 



Problem 1 (Interlock) 



Experimental 
Control 



DID NOT 
COMPLETE 
IN TIME 



19 



COMPLETED 
IN TIME 



34 



20 




X^i = 13.2, p' < .001 



= 16.4, p 



Problem 2 (+100 VDC Power Supply) 



Experimental 
Control 



DID NOT 
COMPLETE 
IN TIME 



13 



28 



COMPLETED 
IN TIME 



26 



11 



X^i = 10.6, p < .001 




x^z = 15.9, p 
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TABLE 3. PERFORMANCE ON FAULT ISOLATION PROBLEMS 
IN SECOND EXAMINATION PERIOD 



Problem 1 (Crystal Detector) 





DID NOT 
COMPLETE 
IN TIME 


COMPLETED 




ERRORS 




IN TIME 




0 


1 


2 or more 


Experimental 


3 


35 




7 


12 


19 


Control 


11 


28 




5 


5 


29 



X^i = 5.3, p < .05 x^2 ~ ^'^^ 
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Opinion Questionnaires, 

All classes held rather favorable attitudes toward the instruction; 
differences betv7een Experimental and Control Groups were negligible (see 
Appendix F, Vol. II). These data are summarized in Table 4. Attitudes 
of the experimental classes toward the particular experimental practices 
were generally rather favorable. On use of media in the classroom all but 
one of the students rated it "somewhat helpful" or "a definite help". 
Similarly, on the paper troubleshooting problems, all but two students 
rated them "somewhat helpful", "helpful", or "very helpful". However, 
attitudes toward the tape/slide demonstrations of checks and adjustments 
were somewhat less favorable: twelve of the students rated them in the 
lower categories of "of little help" or "a waste of time". 
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TA^''^ 4. QUESTIONNAIRE SUMMARY 

The following da^L-^'^^ mean check ratings of how interesting and 
effective the instru^ Was, based upon a nine-point scale with 9 
being most favorable' ^^at-ings are averaged across subjects and across 
the four questions- 



EXPERIMENTAL CONTROL 



CLASSROOM ^ 


5.7 


6.4 


ON STATION 


7.2 


7.1 


(See also App&nd^ 


Vol. I^') 
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DISCUSSION AND INTERPRETATION 



The experimental instructional strategies employing visual simulation, 
in general, produced significantly greater ability to perform job proce- . 
dures in a specified period of testing time, with significantly fewer 
errors, and also resulted in significantly better quiz scores. These 
results were achieved by a typical instructor of USAADS . Primary Instructors, 
with preparation involving a briefing, lesson plan, and a "dry run" for 
practice, V7hich is normal preparation for any new method of instruction. 
The experimental instructional methods also did not require extensive 
devices and they were practicable in the USAADS instructional situation. 

The effects of the various practices are thoroughly confounded in the 
results. However, based upon observations of the process, it seems reason- 
able to attribute most of the results, tentatively, to the classroom acti- 
vities. These practices affected many hours of instruction very noticeably. 
The paper troubleshooting and tape/slide demonstrations of checks and ad- 
justments affected, at most, only a few minutes of instruction. Also, 
performance on the problems was not subject to effective monitoring prac- 
tices, so there was little incentive beyond intrinsic interest in the 
problems, and a rather tenuous relationship to eventual test performance. 
Yet, the students reacted favorably to the problems, according to the 
opinion data, so the fev7 minutes spent on them involves no drawbacks and 
may well be helpful. 

There is a strong tradition in electronics training that relegates 
theory to the classroom and performance to the actual equipment. During 
the development, several iv. .crnctors commented that actual performance 
is not the sort of thing that is taught in the classroom, and there was 
som'itimes confusion concerning whether we were discussing PE 3 (classroom) 
or PE 1 (work with the equipment). Thus theory and practice have come to 
be increasingly separate (in spite of official directives to the contrary). 
The instructional strategies using visual simulation are based upon quite 
a different point of view: that theory and practice are quite compatible, 
that they can and should be taught together, and that visualization is 
actually an important tool for thinking. 
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